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ABSTRACT 

A study was conducted to compare three methods of assessing 
course-level self-efficacy beliefs within a multitrait multimethod (MTMM) 
framework. The methods involved: (1) successfully performing a number of 

domain-related tasks; (2) obtaining specific letter grades in the course; and 
(3) successfully performing generic academic tasks in the context of a 
particular domain. With confirmatory and higher-order confirmatory factor 
analyses, the MTMM approach allowed examining the equivalence of self- 
efficacy responses from different assessment methods (i.e., convergent 
validity) and the distinctiveness of self-efficacy beliefs in different 
academic domains (i.e., discriminant validity). Relations of the three self- 
efficacy measures to other course-specific motivation and performance 
indicators were examined with structural equation modeling. Participants were 
512 middle school students in Korea. Results suggest that different self- 
referent thoughts were called forth depending on whether the course-specific 
self-efficacy items specified different aspects or levels in the expected 
domain performance. When they did, self-efficacy so measured demonstrated 
stronger relations with more cognitive, rather than affective, indicators of 
motivation.' These relations should be studied in more detail 'when achievement 
indexes become, available. (SLD) 
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Compared to other self-theories, self-efficacy research has demonstrated 
relatively consistent operationalizations and unambiguous causal effects on student 
learning (Bong & Clark, 1999; Pajares, 1996). It has greatly benefited from the context- 
specific assessment of constructs (Bandura, 1986, 1997) and has traditionally been 
conducted at task-specific levels. Nevertheless, as research on self-efficacy has 
proliferated, so have the methods of assessing efficacy beliefs. 

Researchers express little disagreement on what constitutes task-specific 
assessment of self-efficacy beliefs. When the target of prediction involves more general 
level outcomes such as course grades, however, it is not always clear which method 
investigators should use to increase the self-efficacy’s predictive power and, at the same 
time, not violate the guidelines suggested by Bandura (1997). Conceivably, when 
researchers’ goal is to predict students’ motivation and performance in given subject 
areas (e.g., mathematics course), they could resort to one of three methods. Investigators 
could assess students’ perceived confidence toward: (1) successfully performing a 
number of major domain-related tasks, (2) obtaining specific letter grades in the course, 
or (3) successfully performing generic academic tasks within the context of a particular 
domain (e.g., “I am sure that I can do an excellent job on the problems and tasks assigned 
for math classes”). 

The purpose of the present study was to compare these 3 methods of assessing 
course-level self-efficacy beliefs within a multi-trait multi-method framework. With 
confirmatory and higher-order confirmatory factor analyses, the MTMM approach 
allowed examining (1) the equivalence of self-efficacy responses from different 
assessment methods (i.e., convergent validity) and (2) the distinctiveness of self-efficacy 
beliefs in different academic domains (i.e., discriminant validity). Further, relations of 
these three self-efficacy measures to other course-specific motivation and performance 
indicators were examined with structural equation modeling. Because three distinct 
subject areas were considered, it was possible to investigate both (1) the relationships of 
different self-efficacy beliefs with other constructs within each domain and (2) 
consistency of these relations across domain. 

Method 

512 middle school students in Seoul, Korea, participated. The motivation survey 
included the following variables: (1) Bandura-type self-efficacy (e.g., “How confident are 
you that you can get a grade better than a B in [a subject]), (2) MSLO self-efficacy 
(Motivated Strategies for Learning Questionnaire; e.g., “I’m certain that I can understand 
what is taught in [a subject] class”) (Pintrich & De Groot, 1990), (3) task-referenced self- 
efficacy (e.g., “How confident are you that you can correctly solve quadratic equation 
problems in math?”), (4) self-efficacy for self-regulated learning (e.g., “How confident 
are you that you can concentrate on your school work?”), (5) self-evaluative standard 
(“What is the lowest grade that you can receive and still be satisfied with in [a subject]?”), 
(6) grade goal (“What academic grade are you striving for in [a subject] course this 
semester?”), (7) perceived value (e.g., “It is important to me to get good grades in [a 
subject]”), and (8) anxiety (e.g.. “I am so nervous during a [subject] test that I cannot 
remember what I have learned”). Principals and teachers of participating schools have 
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agreed to provide the researcher with students’ (9) end-of-semester course grades in the 
domains of interest. These are yet to be received and hence not included in the present 
analyses. With an exception of self-efficacy for self-regulated learning, all variables were 
assessed separately in reference to Korean, Math, and English. Middle school subject 
matter teachers developed the task descriptions included in the task-referenced self- 
efficacy items. 



Results and Discussion 



MTMM Comparison of Scales 

Korean, math, and English self-efficacy were treated as 3 traits. The BDR 
(Bandura-type), MSLQ, and TSE (task-referenced) items were treated as 3 methods. CFA 
models that specified either the trait or method effects only were not able to reproduce the 
data satisfactorily. A model that incorporated the joint effects of trait and method on each 
self-efficacy response displayed good fit to the data, (1091, N = 512) = 2478.852, p 
< .001 (NNFI = .923, CFI= .928). Students’ responses to different self-efficacy items 
were thus affected both by the content domain addressed and the particular method used. 
Students’ self-efficacy beliefs in 3 school subjects were most highly correlated when 
assessed with BDR (average r = .714) and least highly correlated with MSLQ (average r 
= .483). Average correlations among methods were .802 between BDR and MSLQ, .880 
between BDR and TSE, and .823 between MSLQ and TSE across 3 subjects. There was 
clear support for the convergent and discriminant validity according to the Campbell- 
Fiske criteria. 

However, because each of these L‘-order factors reflected combined effects from 
a particular method and a particular subject efficacy, it was not possible to separate the 
relationships among methods from traits and vice versa. A HCFA model allows 
examining the trait correlations after method effects are removed or method correlations 
after trait effects are accounted for, by identifying 2"‘*-order trait and 2"‘*-order method 
factors on the basis of these first-order factors. Significant correlations were observed 
between 2"‘*-order Korean and English (.417), and math and English factors (.374), after 
the method effects were controlled for. The 2"‘*-order BDR and TSE factors correlated 
at .905. However, when the trait effects were removed, the MSLQ factor negatively 
correlated with both the BDR (-.837) and TSE (-.883) factors. The main difference 
between the MSLQ and the other two methods is whether concrete and specific anchors 
for gauging perceived confidence are provided. This difference also manifested itself in 
self-efficacy factors’ relations with other constructs. 

Relations With Other Constructs 

When SEM models with 8 available factors were specified, the following 
relations emerged consistently across 3 subject areas: Self-efficacy for SRL demonstrated 
positive relations with all 3 self-efficacy beliefs and the self-evaluative standard; BDR 
self-efficacy and self-evaluative standard positively predicted students’ grade goal; 

MSLQ self-efficacy showed positive relations with perceived value. Overall, it appears 
that different self-referent thoughts were called forth depending on whether the course- 
specific self-efficacy items specified different aspects or levels in the expected domain 



performance. When they did, self-efficacy so measured demonstrated stronger relations 
with more cognitive, as opposed to affective, indicators of motivation. These relations 
need to be studied further once the achievement indexes become available. 



References 



Bandura, A. (1986). Social foundations of thought and action: A social cognitive 
theory. Englewood Cliffs, NJ: Prentice-Hall. 

Bandura, A. ( 1991 '). Self-efficacv: The exercise of control. New York: Freeman. 

Bong, M., &. Clark, R. E. (1999). Comparison between self-concept and self- 
efficacy in academic motivation research. Educational Psychologist. 34. 139-154. 

Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of 
Educational Research. 66, 543-578. 

Pintrich, P. R., &, De Groot, E. V. (1990). Motivational and self-regulated learning 
components of classroom academic performance. Journal of Educational Psychology, 82. 
33-40. 



ERIC 




♦ ^ 

U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 

Educational Resources Information Center (ERIC) 

REPRODUCTION RELEASE 

(Specific Document) 



® 




TM033732 



I. DOCUMENT IDENTIFICATION: 



Title: 

Evaluation of course-specific self-efficacy assessment methods 

■ ^ "'i. — C: — !lL_ 


AiJ&ior(s): *■' &OK)(r, 


1- 


Corporate Source: 


Publication Date: 

PiPf) ^oo ! 



II. REPRODUCTION RELEASE: 



In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced in the 
monthly abstract journal of the ERIC system. Resources in Education (RIE), are usually made available to users in microfiche, reproduced paper copy 
and electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document and if 
reproduction release is granted, one of the following notices is affixed to the document 



If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottom 
of the page. 



The sample sticker shown below will be 
affixed to alt Level 1 documents 


The sample sticker shown below wiU be 
affixed to all Level 2A documents 


The sample sticker shown below will be 
affixed to all Level 2B documents 


PERMISSION TO REPRODUCE AND 
, DISSE^(IINATE THIS MATERIAL HAS 
-'BEEN granted BY 

r 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE, AND IN ELECTRONIC MEDIA 
FOR ERIC COLLECTION SUBSCRIBERS ONLY. 
HAS BEEN GRANTED BY ^ 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE ONLY HAS BEEN GRANTED BY 


A® 








A® 












TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 


1 




2A 




2B 


Level 1 

r-L^ 


Laval 2A 

I 




Level 2B 

t 




□ 


□ 


Check here for Level 1 release, permitting 
reproduction and dissemination In microfiche or other 
ERiC archival media (e.g., electronic) and paper 
copy. 


Check here for Level 2A release, permitting 
reproduction and dissemination in microfiche and in 
electronic media for ERIC wchival collection 
subscrtbers only 


Check here for Level 2B release, permitting 
reproduction and dissemination in microfiche only 



Documents will bo processed os indicated provided reproduction quality pennita. 

If penoissioo to reproduce Is granted, but no box is checked, dooanents will be processed at Level 1 , 



/ hereby granf to the Educational Resources Information Canter (ERIC) nonexdusive permission to reproduce and disseminate this document 
as Qtiove. Reproduction from the ERIC microfiche or electronic media by persons other then ERIC employees end its system 

(^ntractors r^uires permission from the copyright holder. Exception is made fornon-profit reproduction by libraries end other ser\/ice agencie s 
to satisfy information needs of educators in response to discrete inquiries. 


Sigr-atuf«; 

/Qryry' 


fyrrA. 


Organctation/Addrsss: / ^ 

Mimi Bong, Ph D 
Oef^ofEd Psychology 
Univ of South Carolina 




Fax 


E-Mail Addrw; 


Date. ' 

: 3 A 



Sign 

here,-^ 

please 






(over) 




♦ 



ill. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE): 

If permission to reproduce is not granted to ERIC, or, if you wish ERIC to dte the availability of the document from another source, please 
provide the following information regarding the availability of the document. (ERIC will not announce a document unless it is publicly 
available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria are significantly more 
stringent for documents that cannot be made available through EDRS.) 




IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER: 

If the right to grant this reproduction release is held by someone other than the addressee, please provide the appropriate name and 

sHHrocc* * * * 




V. WHERE TO SEND THIS FORM: 



Send this form to the following ERIC Clearinghouse: 

ERIC CLEARINGHOUSE ON ASSESSMENT AND EVALUATION 
UNIVERSITY OF MARYLAND 
1129 SHRIVERLAB 
COLLEGE PARK, MD 20742-5701 
ATTN: ACQUISITIONS 



However, if solicited by the ERIC Facility, or if making an unsolicited contribution 
contributed) to: 



to ERIC, return this form (and the document being 



ERIC Processing and Reference facility 
4483-A Forbes Boulevard 
Lanham, Maryland 20706 

Telephone: 301-552-4200 
Toll Free: 800-799-3742 
FAX: 301-552-4700 

^ e-mail: ericfac@ineted.gov 

ERlCFF^iaa (R.,. MOOD ****• 



